On boosting kernel regression
نویسندگان
چکیده
In this paper we propose a simple multistep regression smoother which is constructed in an iterative manner, by learning the Nadaraya-Watson estimator with L2boosting. We find, in both theoretical analysis and simulation experiments, that the bias converges exponentially fast, and the variance diverges exponentially slow. The first boosting step is analyzed in more detail, giving asymptotic expressions as functions of the smoothing parameter, and relationships with previous work are explored. Practical performance is illustrated by both simulated and real data.
منابع مشابه
L 2 Boosting in Kernel Regression 1
In this paper, we investigate the theoretical and empirical properties of L2 boosting with kernel regression estimates as weak learners. We show that each step of L2 boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that L2 boosting...
متن کاملL2 boosting in kernel regression
In this paper, we investigate the theoretical and empirical properties of L2 boosting with kernel regression estimates as weak learners. We show that each step of L2 boosting reduces the bias of the estimate by two orders of magnitude, while it does not deteriorate the order of the variance. We illustrate the theoretical findings by some simulated examples. Also, we demonstrate that L2 boosting...
متن کاملSparse Regression Modelling Using an Incremental Weighted Optimization Method Based on Boosting with Correlation Criterion
ABSTRACT A novel technique is presented to construct sparse Gaussian regression models. Unlike most kernel regression modelling methods, which restrict kernel means to the training input data and use a fixed common variance for all the regressors, the proposed technique can tune the mean vector and diagonal covariance matrix of individual Gaussian regressor to best fit the training data based o...
متن کاملBoosting Based Multiple Kernel Learning and Transfer Regression for Electricity Load Forecasting
Accurate electricity load forecasting is of crucial importance for power system operation and smart grid energy management. Different factors, such as weather conditions, lagged values, and day types may affect electricity load consumption. We propose to use multiple kernel learning (MKL) for electricity load forecasting, as it provides more flexibilities than traditional kernel methods. Comput...
متن کاملMultistep kernel regression smoothing by boosting
In this paper we propose a simple multistep regression smoother which is constructed in a boosting fashion, by learning the Nadaraya–Watson estimator with L2Boosting. Differently from the usual approach, we do not focus on L2Boosting for ever. Given a kernel smoother as a learner, we explore the boosting capability to build estimators using a finite number of boosting iterations. This approach ...
متن کامل